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DETAILED ACTION 

1 . This Office Action is in response to an original application filed 1 0/03/2003 with a 
priority date of 10/03/2003. 

2. Claims 1-65 are currently pending. Claims 1, 22, and 48 are independent claims. 

Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

4. Claims 1 , 2, 4, and 48-50 are rejected under 35 U.S.C. 102(b) as being 
anticipated by Yamashita et al. (hereinafter Yamashita, U.S. Patent No. 5,555,362 filed 
07/26/1995, issued 09/10/1996). 

In regard to independent Claim 1 (and similarly independent Claim 48), 
Yamashita teaches receiving a definition of at least one region in an image in that a 
document image is scanned by the image input unit (2), and character strings, vertical 
and horizontal black lines, and other black pixel regions (picture-element) are extracted 
from the image and stored in the image memory 3 (Step 21) (Col. 3, lines 35-40). 

Yamashita also teaches the region definition having a location specification and a 
type specification in that subsequent processing is executed in accordance with 
extracted rectangle data. Then, area segmentation of the document image is 
automatically executed by the automatic area segmentation unit 4A of the area 
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generation unit 4 (Step.22). First, long, wide, and White pixel regions (type specification) 
and long black lines (type specification) to serve as separators for objects are extracted 
from the x,y-coordinates of the rectangle (position). Then, graphic areas (type 
specification) are removed before character areas (type specification) are roughly 
segmented using the extracted separator (Col. 3, lines 41-49). 

Yamashita also teaches displaying the boundaries of the at least one defined 
region according to its type specification (see Fig. 3). 

Yamashita also teaches receiving a definition of a visible area in the image, the 
visible area definition having a specification of margins around the image; and 
generating an image layout definition comprising the region definition and the visible 
area definition (see Fig 6, layout created from segmentation analysis of image positions 
and components). 

In regard to dependent Claim 2, Yamashita teaches displaying the image on a 
display (Col. 3, lines 53-59; Fig. 6). 

In regard to dependent Claim 49, Claim 49 contains subject matter similar to 
that found in Claim 1 (and similarly Claim 48), and is rejected along similar lines of 
reasoning. 

In regard to dependent Claim 4 (and similarly independent Claim 50), 

Yamashita teaches automatically determining the definition of the at least one region in 
the image by segmentation analysis of the image (see Figs. 2 and 3, Fig. 2 step 22). 
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Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

6. Claims 12-14, and 58-60 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Yamashita. 

In regard to dependent Claim 12-14 (and similarly dependent Claims 58-60), 

Yamashita fails to teach such limitations for manipulating scanned document images for 
purposes of displaying or transmitting in order to provide an image that is the 
appropriate size, dimension, color depth for the given action. However, such functions 
were known and obvious to one of ordinary skill in the art at the time of invention 
particularly with respect to graphical user interfaces where one would have desired to 
view entire images on a screen independent of the size of the actual image for purposes 
such as identifying regions of interest. 



Application/Control Number: 10/679,154 



Art Unit: 2176 



Page 5 



7. Claims 3, 11, 17, and 57 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Yamashita in view of Revankar et al. (hereinafter Revankar, U.S. 
Patent No. 5,767,978 filed 01/21/1997, issued 06/16/1998). 

In regard to dependent Claim 3, Yamashita fails to teach receiving a definition 
of at least one region in an image further comprises receiving a modality specification. 
However, Revankar teaches image segmentation according to classes of regions that 
may be rendered according to the same imaging techniques. Image regions may be 
rendered according to a three-class system (such as traditional text, graphic and picture 
systems), or according to more than three image classes. In addition, only two image 
classes may be required to render high quality draft or final output images. The image 
characteristics that may be rendered differently from class to class may include half 
toning, colorization and other image attributes (see Abstract). It would have been 
obvious to one of ordinary skill in the art at the time of invention to combine the 
teachings of Yamashita and Revankar as both inventions relate to image segmentation. 
Adding the teaching of Revankar provides the benefit of recognizing region types by 
class and by modality (color, bit depth, etc.). 

In regard to dependent Claim 11 (and similarly dependent Claim 57), 
Yamashita fails to teach receiving a user input indicative of a first vertex and a location 
of a second vertex opposite the first vertex of the visible area on the image. However, 
Revankar teaches image segmentation according to classes of regions that may be 
rendered according to the same imaging techniques. Image regions may be rendered 
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according to a three-class system (such as traditional text, graphic and picture 
systems), or according to more than three image classes. In addition, only two image 
classes may be required to render high quality draft or final output images. The image 
characteristics that may be rendered differently from class to class may include half 
toning, colorization and other image attributes (see Abstract). It would have been 
obvious to one of ordinary skill in the art at the time of invention to combine the 
teachings of Yamashita and Revankar as both inventions relate to image segmentation. 
Adding the teaching of Revankar provides the benefit of recognizing region types by 
class and by modality (color, bit depth, etc.). 

In regard to dependent Claim 17, Yamashita fails to teach receiving user 
specification of region type and region modality. However, Revankar teaches image 
segmentation according to classes of regions that may be rendered according to the 
same imaging techniques. Image regions may be rendered according to a three-class 
system (such as traditional text, graphic and picture systems), or according to more 
than three image classes. In addition, only two image classes may be required to render 
high quality draft or final output images. The image characteristics that may be rendered 
differently from class to class may include halftoning, colorization and other image 
attributes (see Abstract). It would have been obvious to one of ordinary skill in the art at 
the time of invention to combine the teachings of Yamashita and Revankar as both 
inventions relate to image segmentation. Adding the teaching of Revankar provides the 
benefit of recognizing region types by class and by modality (color, bit depth, etc.). 
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8. Claims 5, and 51 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Yamashita in view of Sakai et al. (hereinafter Sakai, U.S. Patent No. 6,735,740 
filed 03/04/1998, issued 05/11/2004). 

In regard to dependent Claim 5 (and similarly independent Claim 51), 
Yamashita fails to teach automatically determining the definition of the at least one 
region in the image by classification analysis of the image. However, Sakai teaches 
such a limitation (Figs. 10A-C; depict progressive classification of image regions based 
on type). It would have been obvious to one of ordinary skill in the art at the time of 
invention to combine the teachings of Yamashita and Sakai as both inventions relate to 
document image analysis. Adding the teaching of Sakai provides the benefit of 
partitioning an image based on types of content identified in the image. 

9. Claims 6, and 52 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Yamashita in view of Ohta (U.S. Patent No. 6,163,623 filed 07/26/1995, issued 
12/19/2000). 

In regard to dependent Claim 6 (and similarly dependent Claim 52), 

Yamashita fails to teach receiving a user input indicative of a point on the image; and 
defining a region encompassing the point using segmentation and classification 
analyses of the image. However, Ohta teaches scanning a document, rendering it to a 
touch display, and allowing the user to manually select a region or regions to further 
process; the drawing of a box is done automatically (Col. 7, lines 46-67; Col. 8, lines 1- 
2). It would have been obvious to one of ordinary skill in the art at the time of invention 
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to combine the teachings of Yamashita and Ohta as both inventions relate to 
designating regions of documents for further analysis. Adding the teaching of Ohta 
provides the user with a means to easily choose which portions of a document to further 
analyze. 

* 

10. Claims 7, 15, and 53 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Yamashita in view of Rangarajan (U.S. Patent No. 5,822,454 filed 04/10/1995, 
issued 10/13/1998). 

In regard to dependent Claim 7 (and similarly dependent Claim 53), 
Yamashita fails to teach receiving a user input indicative of boundaries of the region on 
the image; and receiving a user input indicative of region type and region modality 
specifications. However, Ranaaraian teaches a conventional set of drawing-like tools 
with which the user can graphically create 31 1 the user defined zones. This is done by 
choosing an appropriate drawing tool, such as a rectangle or polygon creation tool, and 
applying it to the de-skewed image to select the individual areas or zones containing the 
desired text information. Fig. 7a illustrates one example of a suitable user interface 705, 
showing a de-skewed document 700. Fig. 7b illustrates the same document now 
including a number of user-defined zones 701. A palette of drawing tools 703 is also 
shown, with various graphical tools for selecting the user-defined zones 701. Once the 
user defines a number of zones, the coordinates of the boundary of each of user 
defined zone is stored, preferably using the coordinates of an upper left hand corner, 
and a lower right hand corner where the user defined zone is a rectangle. For general 
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polygonal user defined zones, the coordinates of each vertex may also be stored (Col. 
9, lines 15-37; Figs. 7A-B). It would have been obvious to one of ordinary skill in the art 
at the time of invention to combine the teachings of Yamashita and Ranqaraian as both 
inventions relate to document image analysis. Adding the teaching of Ranqaraian 
provides the benefit of manually defining image regions. 

In regard to dependent Claim 15, Yamashita fails to teach receiving definition 
of at least one region comprises receiving a user specification of a location and 
boundaries of a region in the image. However, Ranqaraian teaches input of vertices to 
define an image region (Col. 9, lines 15-37; inputting polygons). It would have been 
obvious to one of ordinary skill in the art at the time of invention to combine the 
teachings of Yamashita and Ranqaraian as both inventions relate to document image 
analysis. Adding the teaching of Ranqaraian provides the benefit of manually defining 
image regions. 
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1 1 . Claims 8-1 0, and 54-56 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Yamashita in view of Rangarajan, and in further view of Revankar. 

' In regard to dependent Claim 8 (and similarly dependent Claim 54), 
Yamashita fails to teach receiving a user input indicative of vertices of the region on the 
image. However, Rangarajan teaches input of vertices to define an image region (Col. 
9, lines 15-37). It would have been obvious to one of ordinary skill in the art at the time 
of invention to combine the teachings of Yamashita and Rangarajan as both inventions 
relate to document image analysis. Adding the teaching of Rangarajan provides the 
benefit of manually defining image regions. 

Yamashita also fails to teach receiving a user input indicative of region type and 
region modality specifications. However, Revankar teaches image segmentation 
according to classes of regions that may be rendered according to the same imaging 
techniques. Image regions may be rendered according to a three-class system (such as 
traditional text, graphic and picture systems), or according to more than three image 
classes. In addition, only two image classes may be required to render high quality draft 
or final output images. The image characteristics that may be rendered differently from 
class to class may include halftoning, colorization and other image attributes (see 
Abstract). It would have been obvious to one of ordinary skill in the art at the time of 
invention to combine the teachings of Yamashita and Revankar as both inventions 
relate to image segmentation. Adding the teaching of Revankar provides the benefit of 
recognizing region types by class and by modality (color, bit depth, etc.). 
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In regard to dependent Claim 9 (and similarly dependent Claim 55), 

Yamashita teaches receiving a user input indicative of vertices of a polygonal region on 
the image. However, Rangaraian teaches input of vertices to define an image region 
(Col. 9, lines 15-37). It would have been obvious to one of ordinary skill in the art at the 
time of invention to combine the teachings of Yamashita and Rangaraian as both 
inventions relate to document image analysis. Adding the teaching of Rangaraian 
provides the benefit of manually defining image regions. 

Yamashita also fails to teach receiving a user input indicative of region type and 
region modality specifications of the polygonal region. However, Revankar teaches 
image segmentation according to classes of regions that may be rendered according to 
the same imaging techniques. Image regions may be rendered according to a three- 
class system (such as traditional text, graphic and picture systems), or according to 
more than three image classes. In addition, only two image classes may be required to 
render high quality draft or final output images. The image characteristics that may be 
rendered differently from class to class may include halftoning, colorization and other 
image attributes (see Abstract). It would have been obvious to one of ordinary skill in 
the art at the time of invention to combine the teachings of Yamashita and Revankar as 
both inventions relate to image segmentation. Adding the teaching of Revankar 
provides the benefit of recognizing region types by class and by modality (color, bit 
depth, etc.). 

In regard to dependent Claim 10 (and similarly dependent Claim 56), 

Yamashita fails to teach receiving a user input indicative of a first vertex and a location 
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of a second vertex opposite the first vertex of a rectangular region on the image. 
However, Ranaaraian teaches input of vertices to define an image region (Col. 9, lines 
15-37). It would have been obvious to one of ordinary skill in the art at the time of 
invention to combine the teachings of Yamashita and Rangaraian as both inventions 
relate to document image analysis. Adding the teaching of Rangaraian provides the 
benefit of manually defining image regions. 

Yamashita also fails to teach receiving a user input indicative of region type and 
region modality specifications of the rectangular region. However, Revankar teaches 
image segmentation according to classes of regions that may be rendered according to 
the same imaging techniques. Image regions may be rendered according to a three- 
class system (such as traditional text, graphic and picture systems), or according to 
more than three image classes. In addition, only two image classes may be required to 
render high quality draft or final output images. The image characteristics that may be 
rendered differently from class to class may include halftoning, colorization and other 
image attributes (see Abstract). It would have been obvious to one of ordinary skill in 
the art at the time of invention to combine the teachings of Yamashita and Revankar as 
both inventions relate to image segmentation. Adding the teaching of Revankar 
provides the benefit of recognizing region types by class and by modality (color, bit 
depth, etc.). 
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12. Claims 16, 20, 61, and 64 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Yamashita in view of Rangarajan, and in further view of Mahoney et 
al. (hereinafter Mahoney, U.S. Patent No. 5,999,664 filed 11/14/1997, issued 
12/07/1999). 

In regard to dependent Claim 16, Yamashita teaches receiving definition of at 
least one region comprises verifying the user-specified region location and boundaries 
conform to at least one region management model. However, Mahoney teaches 
searching and identifying documents based on their makeup (structure, content, etc.). 
Their system performs structural analysis at two levels. At the lower level, specific 
layout formats of a document can be identified (e.g., the recipient field of a letter or the 
header field of a memo). Such identification is performed herein using features. At the 
higher level, the entire configuration of an input document is captured using genre 
models. For example, a "business letter" is a genre model of a document that can be 
defined in most instances by a letter-date feature, a letter-recipient feature, a letter-cc 
feature, and a letter-signature feature (as shown in Fig. 3). Although some models may 
have some features in common, such models may still be distinguishable from each 
other by either the presence or absence of other features. For example, a memo and a 
letter may have similar letter-signature features while each may have different 
document header features (e.g., four-memo mark and letter-recipient). It would have 
been obvious to one of ordinary skill in the art at the time of invention to combine the 
teachings of Yamashita and Mahoney as both inventions relate to comparing document 
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images to models or templates of documents. Adding the teaching of Mahonev provides 
the benefit of identifying documents (or regions thereof) with models. 

In regard to dependent Claim 20 (and similarly dependent Claim 64), 
Yamashita fails to teach determining whether the user-specified region boundaries fall 
within the visible area. However, Mahonev teaches searching and identifying 
documents based on their makeup (structure, content, etc.). Their system performs 
structural analysis at two levels. At the lower level, specific layout formats of a 
document can be identified (e.g., the recipient field of a letter or the header field of a 
memo). Such identification is performed herein using features. At the higher level, the 
entire configuration of an input document is captured using genre models. For example, 
a "business letter" is a genre model of a document that can be defined in most 
instances by a letter-date feature, a letter-recipient feature, a letter-cc feature, and a 
letter-signature feature (as shown in Fig. 3). Although some models may have some 
features in common, such models may still be distinguishable from each other by either 
the presence or absence of other features. For example, a memo and a letter may have 
similar letter-signature features while each may have different document header 
features (e.g., four-memo mark and letter-recipient). It would have been obvious to one 
of ordinary skill in the art at the time of invention to combine the teachings of Yamashita 
and Mahonev as both inventions relate to comparing document images to models or 
templates of documents. Adding the teaching of Mahonev provides the benefit of 
identifying documents (or regions thereof) with models. 
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In regard to dependent Claim 61, Claim 61 contains subject matter similar to 
that found in Claims 15 and 16, and is rejected along similar lines of reasoning. 

13. Claims 18-19, 62-63 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Yamashita in view of Rangarajan, and in further view of Mahoney, and in further 
view of Taylor et al. (hereinafter Taylor, U.S. Patent No. 5,848,184 filed 06/30/1995, 
issued 12/08/1998). 

In regard to dependent Claim 18-19 (and similarly dependent Claims 62-63), 

Yamashita fails to teach determining whether the user-specified region boundaries 
overlap with another region. However, Taylor teaches detection of overlapping 
boundaries as well as bounding boxes, which cross one another (Col. 7, lines 36-63). 

14. Claims 21 , and 65 are rejected under 35 U.S.C. 1 03(a) as being unpatentable 
over Yamashita in view of Rangarajan, and in further view of Ahlstrom et al. (hereinafter 
Ahlstrom, U.S. Patent No. 6,594,030 filed 08/27/1999, issued 07/15/2003). 

In regard to dependent Claim 21 (and similarly dependent Claims 42, and 
65), Yamashita fails to teach determining whether the user-specified region comply with 
a predetermined multiple z-order specification. However, Ahlstrom teaches z-order as it 
relates to how pages are overlapped upon one another (Col. 6, lines 23-56). It would 
have been obvious to one of ordinary skill in the art at the time of invention to combine 
the teachings of Yamashita and Ahlstrom as both inventions relate to analysis of page 
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objects. Adding the teaching of Ahlstrom provides the benefit of checking z-ordering of 
pages. 

1 5. Claims 22-24, 33-35, and 43-47 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Yamashita in view of Mahoney. 

In regard to independent Claim 22, Claim 22 reflects the method of Claim 1 
and is rejected along the same rationale. 

In addition, Yamashita fails to teach searching for an image layout definition 
template that best matches the generated image layout definition; and conforming the 
generated image layout definition to the best-matched image layout definition template. 
However, Mahonev teaches searching and identifying documents based on their 
makeup (structure, content, etc.). Their system performs structural analysis at two 
levels. At the lower level, specific layout formats of a document can be identified (e.g., 
the recipient field of a letter or the header field of a memo). Such identification is 
performed herein using features. At the higher level, the entire configuration of an input 
document is captured using genre models. For example, a "business letter" is a genre 
model of a document that can be defined in most instances by a letter-date feature, a 
letter-recipient feature, a letter-cc feature, and a letter-signature feature (as shown in 
Fig. 3). Although some models may have some features in common, such models may 
still be distinguishable from each other by either the presence or absence of other 
features. For example, a memo and a letter may have similar letter-signature features 
while each may have different document header features (e.g., four-memo mark and 



Application/Control Number: 1 0/679, 1 54 Page 1 7 

Art Unit: 2176 

letter-recipient). It would have been obvious to one of ordinary skill in the art at the time 
of invention to combine the teachings of Yamashita and Mahonev as both inventions 
relate to comparing document images to models or templates of documents. Adding the 
teaching of Mahonev provides the benefit of identifying documents (or regions thereof) 
with models. 

In regard to dependent Claim 23, Yamashita teaches displaying the image on a 
display (Col. 3, lines 53-59; Fig. 6). 

In regard to dependent Claim 24, Claim 24 reflects the method as claimed in 
Claim 1 (and similarly Claim 48), and is rejected along the same rationale. 

In regard to dependent Claims 33-35, Yamashita fails to teach such limitations 
for manipulating scanned document images for purposes of displaying or transmitting in 
order to provide an image that is the appropriate size, dimension, color depth for the 
given action. However, such functions were known and obvious to one of ordinary skill 
in the art at the time of invention particularly with respect to graphical user interfaces 
where one would have desired to view entire images on a screen independent of the 
size of the actual image for purposes such as identifying regions of interest. 

In regard to dependent Claims 43-45, and 47, Yamashita fails to explicitly 
teach adjusting the location, type, modality, or visible area specification of the at least 
one region of the image layout definition. However Mahonev teaches a document 
search system provides a user with a programming interface for dynamically specifying 
features of documents recorded in a corpus of documents (Abstract). Mahoney provides 
a user interface which allows for the definition or adjustment of a given documents' 
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parameters in order to search a corpus of documents looking for similarities. Thus, it 
would have been obvious to one of ordinary skill in the art at the time of invention to use 
the user interface of Mahoney to make adjustments in the model of a current document 
to make identification of all or a part of similar documents more likely. It also would have 
been obvious to one of ordinary skill in the art at the time of invention to combine the 
teachings of Yamashita and Mahoney as both inventions relate to comparing document 
images to models or templates of documents. Adding the teaching of Mahoney provides 
the benefit of identifying documents (or regions thereof) with models. 

In regard to dependent Claim 46, Claim 46 contains subject matter similar to 
that found in Claim 1 (and similarly Claim 48), and is rejected along similar lines of 
reasoning. 

16. Claims 25 and 32 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Yamashita in view of Mahoney, and in further view of Revankar. 

In regard to dependent Claim 25, Yamashita fails to teach receiving a definition 
of at least one region in an image further comprises receiving a modality specification. 
However, Revankar teaches image segmentation according to classes of regions that 
may be rendered according to the same imaging techniques. Image regions may be 
rendered according to a three-class system (such as traditional text, graphic and picture 
systems), or according to more than three image classes. In addition, only two image 
classes may be required to render high quality draft or final output images. The image 
characteristics that may be rendered differently from class to class may include half 
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toning, colorization and other image attributes (see Abstract). It would have been 
obvious to one of ordinary skill in the art at the time of invention to combine the 
teachings of Yamashita and Revankar as both inventions relate to image segmentation. 
Adding the teaching of Revankar provides the benefit of recognizing region types by 
class and by modality (color, bit depth, etc.). 

In regard to dependent Claim 32, Yamashita fails to teach receiving a user 
input indicative of a first vertex and a location of a second vertex opposite the first 
vertex of the visible area on the image. However, Revankar teaches image 
segmentation according to classes of regions that may be rendered according to the 
same imaging techniques. Image regions may be rendered according to a three-class 
system (such as traditional text, graphic and picture systems), or according to more 
than three image classes. In addition, only two image classes may be required to render 
high quality draft or final output images. The image characteristics that may be rendered 
differently from class to class may include halftoning, colorization and other image 
attributes (see Abstract). It would have been obvious to one of ordinary skill in the art at 
the time of invention to combine the teachings of Yamashita and Revankar as both 
inventions relate to image segmentation. Adding the teaching of Revankar provides the 
benefit of recognizing region types by class and by modality (color, bit depth, etc.). 
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17. Claim 26 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Yamashita in view of Mahoney, and in further view of Sakai. 

In regard to dependent Claim 26, Claim 26 reflects the method of Claims 4 and 
5 and is rejected along the same rationale. 

18. Claim 27 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Yamashita in view of Mahoney, and in further view of Ohta. 

In regard to dependent Claim 27, Yamashita fails to teach receiving a user 
input indicative of a point on the image; and defining a region encompassing the point 
using segmentation and classification analyses of the image. However, Ohta teaches 
scanning a document, rendering it to a touch display, and allowing the user to manually 
select a region or regions to further process; the drawing of a box is done automatically 
(Col. 7, lines 46-67; Col. 8, lines 1-2). It would have been obvious to one of ordinary skill 
in the art at the time of invention to combine the teachings of Yamashita and Ohta as 
both inventions relate to designating regions of documents for further analysis. Adding 
the teaching of Ohta provides the user with a means to easily choose which portions of 
a document to further analyze. 
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1 9. Claims 28, 36-37, and 41 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Yamashita in view of Mahoney, and in further view of Rangarajan. 

In regard to dependent Claim 28, Yamashita fails to teach receiving a user 
input indicative of boundaries of the region on the image; and receiving a user input 
indicative of region type and region modality specifications. However, Rangarajan 
teaches a conventional set of drawing-like tools with which the user can graphically 
create 31 1 the user defined zones. This is done by choosing an appropriate drawing 
tool, such as a rectangle or polygon creation tool, and applying it to the de-skewed 
image to select the individual areas or zones containing the desired text information. 
Fig. 7a illustrates one example of a suitable user interface 705, showing a de-skewed 
document 700. Fig. 7b illustrates the same document now including a number of user- 
defined zones 701. A palette of drawing tools 703 is also shown, with various graphical 
tools for selecting the user-defined zones 701 . Once the user defines a number of 
zones, the coordinates of the boundary of each of user defined zone is stored, 
preferably using the coordinates of an upper left hand corner, and a lower right hand 
corner where the user defined zone is a rectangle. For general polygonal user defined 
zones, the coordinates of each vertex may also be stored (Col. 9, lines 15-37; Figs. 7A- 
B). It would have been obvious to one of ordinary skill in the art at the time of invention 
to combine the teachings of Yamashita and Rangarajan as both inventions relate to 
document image analysis. Adding the teaching of Rangarajan provides the benefit of 
manually defining image regions. 
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In regard to dependent Claim 36, Yamashita fails to teach receiving definition 
of at least one region comprises receiving a user specification of a location and 
boundaries of a region in the image. However, Ranqaraian teaches input of vertices to 
define an image region (Col. 9, lines 15-37). It would have been obvious to one of 
ordinary skill in the art at the time of invention to combine the teachings of Yamashita 
and Ranqaraian as both inventions relate to document image analysis. Adding the 
teaching of Ranqaraian provides the benefit of manually defining image regions. 

In regard to dependent Claim 37, Yamashita teaches receiving definition of at 
least one region comprises verifying the user-specified region location and boundaries 
conform to at least one region management model. However, Mahonev teaches 
searching and identifying documents based on their makeup (structure, content, etc.). 
Their system performs structural analysis at two levels. At the lower level, specific 
layout formats of a document can be identified (e.g., the recipient field of a letter or the 
header field of a memo). Such identification is performed herein using features. At the 
higher level, the entire configuration of an input document is captured using genre 
models. For example, a "business letter" is a genre model of a document that can be 
defined in most instances by a letter-date feature, a letter-recipient feature, a letter-cc 
feature, and a letter-signature feature (as shown in Fig. 3). Although some models may 
have some features in common, such models may still be distinguishable from each 
other by either the presence or absence of other features. For example, a memo and a 
letter may have similar letter-signature features while each may have different 
document header features (e.g., four-memo mark and letter-recipient). It would have 
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been obvious to one of ordinary skill in the art at the time of invention to combine the 
teachings of Yamashita and Mahonev as both inventions relate to comparing document 
images to models or templates of documents. Adding the teaching of Mahonev provides 
the benefit of identifying documents (or regions thereof) with models. 

In regard to dependent Claim 41, Yamashita fails to teach determining whether 
the user-specified region boundaries fall within the visible area. However, Mahonev 
teaches searching and identifying documents based on their makeup (structure, 
content, etc.). Their system performs structural analysis at two levels. At the lower level, 
specific layout formats of a document can be identified (e.g., the recipient field of a letter 
or the header field of a memo). Such identification is performed herein using features. 
At the higher level, the entire configuration of an input document is captured using 
genre models. For example, a "business letter" is a genre model of a document that can 
be defined in most instances by a letter-date feature, a letter-recipient feature, a letter- 
cc feature, and a letter-signature feature (as shown in Fig. 3). Although some models 
may have some features in common, such models may still be distinguishable from 
each other by either the presence or absence of other features. For example, a memo 
and a letter may have similar letter-signature features while each may have different 
document header features (e.g., four-memo mark and letter-recipient). It would have 
been obvious to one of ordinary skill in the art at the time of invention to combine the 
teachings of Yamashita and Mahonev as both inventions relate to comparing document 
images to models or templates of documents. Adding the teaching of Mahonev provides 
the benefit of identifying documents (or regions thereof) with models. 
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20. Claims 29-31 , and 38 are rejected under 35 U.S.C. 1 03(a) as being unpatentable 
over Yamashita in view of Rangarajan, and in further view of Revankar. 

In regard to dependent Claim 29, Yamashita fails to teach receiving a user 
input indicative of vertices of the region on the image. However, Rangarajan teaches 
input of vertices to define an image region (Col. 9, lines 15-37). It would have been 
obvious to one of ordinary skill in the art at the time of invention to combine the 
teachings of Yamashita and Rangarajan as both inventions relate to document image 
analysis. Adding the teaching of Rangarajan provides the benefit of manually defining 
image regions. 

Yamashita also fails to teach receiving a user input indicative of region type and 
region modality specifications. However, Revankar teaches image segmentation 
according to classes of regions that may be rendered according to the same imaging 
techniques. Image regions may be rendered according to a three-class system (such as 
traditional text, graphic and picture systems), or according to more than three image 
classes. In addition, only two image classes may be required to render high quality draft 
or final output images. The image characteristics that may be rendered differently from 
class to class may include half toning, colorization and other image attributes (see 
Abstract). It would have been obvious to one of ordinary skill in the art at the time of 
invention to combine the teachings of Yamashita and Revankar as both inventions 
relate to image segmentation. Adding the teaching of Revankar provides the benefit of 
recognizing region types by class and by modality (color, bit depth, etc.). 
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In regard to dependent Claim 30, Yamashita teaches receiving a user input 
indicative of vertices of a polygonal region on the image. However, Ranqaraian teaches 
input of vertices to define an image region (Col. 9, lines 15-37). It would have been 
obvious to one of ordinary skill in the art at the time of invention to combine the 
teachings of Yamashita and Ranqaraian as both inventions relate to document image 
analysis. Adding the teaching of Ranqaraian provides the benefit of manually defining 
image regions. 

Yamashita also fails to teach receiving a user input indicative of region type and 
region modality specifications of the polygonal region. However, Revankar teaches 
image segmentation according to classes of regions that may be rendered according to 
the same imaging techniques. Image regions may be rendered according to a three- 
class system (such as traditional text, graphic and picture systems), or according to 
more than three image classes. In addition, only two image classes may be required to 
render high quality draft or final output images. The image characteristics that may be 
rendered differently from class to class may include halftoning, colorization and other 
image attributes (see Abstract). It would have been obvious to one of ordinary skill in 
the art at the time of invention to combine the teachings of Yamashita and Revankar as 
both inventions relate to image segmentation. Adding the teaching of Revankar 
provides the benefit of recognizing region types by class and by modality (color, bit 
depth, etc.). 

In regard to dependent Claim 31, Yamashita fails to teach receiving a user 
input indicative of a first vertex and a location of a second vertex opposite the first 
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vertex of a rectangular region on the image. However, Ranqaraian teaches input of 
vertices to define an image region (Col. 9, lines 15-37). It would have been obvious to 
one of ordinary skill in the art at the time of invention to combine the teachings of 
Yamashita and Ranqaraian as both inventions relate to document image analysis. 
Adding the teaching of Ranqaraian provides the benefit of manually defining image 
regions. 

Yamashita also fails to teach receiving a user input indicative of region type and 
region modality specifications of the rectangular region. However, Revankar teaches 
image segmentation according to classes of regions that may be rendered according to 
the same imaging techniques. Image regions may be rendered according to a three- 
class system (such as traditional text, graphic and picture systems), or according to 
more than three image classes. In addition, only two image classes may be required to 
render high quality draft or final output images. The image characteristics that may be 
rendered differently from class to class may include half toning, colorization and other 
image attributes (see Abstract). It would have been obvious to one of ordinary skill in 
the art at the time of invention to combine the teachings of Yamashita and Revankar as 
both inventions relate to image segmentation. Adding the teaching of Revankar 
provides the benefit of recognizing region types by class and by modality (color, bit 
depth, etc.). 

In regard to dependent Claim 38, Yamashita fails to teach receiving user 
specification of region type and region modality. However, Revankar teaches image 
segmentation according to classes of regions that may be rendered according to the 
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same imaging techniques. Image regions may be rendered according to a three-class 
system (such as traditional text, graphic and picture systems), or according to more 
than three image classes. In addition, only two image classes may be required to render 
high quality draft or final output images. The image characteristics that may be rendered 
differently from class to class may include half toning, colorization and other image 
attributes (see Abstract). It would have been obvious to one of ordinary skill in the art at 
the time of invention to combine the teachings of Yamashita and Revankar as both 
inventions relate to image segmentation. Adding the teaching of Revankar provides the 
benefit of recognizing region types by class and by modality (color, bit depth, etc.). 

21 . Claims 39-40 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Yamashita in view of Mahoney, and in further view of Rangarajan, and in further view of 
Taylor. 

In regard to dependent Claim 39-40, Yamashita fails to teach determining 
whether the user-specified region boundaries overlap with another region. However, 
Taylor teaches detection of overlapping boundaries as well as bounding boxes, which 
cross one another (Col. 7, lines 36-63). 
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22. Claim 42 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Yamashita in view of Mahoney, and in further view of Rangarajan, and in further view of 
Ahlstrom. 

In regard to dependent Claim 42, Yamashita fails to teach determining whether 
the user-specified region comply with a predetermined multiple z-order specification. 
However, Ahlstrom teaches z-order as it relates to how pages are overlapped upon one 
another (Col. 6, lines 23-56). It would have been obvious to one of ordinary skill in the 
art at the time of invention to combine the teachings of Yamashita and Ahlstrom as both 
inventions relate to analysis of page objects. Adding the teaching of Ahlstrom provides 
the benefit of checking z-ordering of pages. 
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Conclusion 



23. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to James H. Blackwell whose telephone number is 571- 

272- 4089. The examiner can normally be reached on Mon-Fri. 

24. If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Heather R. Herndon can be reached on 571-272-4136. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 

273- 8300. 

25. Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for published 
applications may be obtained from either Private PAIR or Public PAIR. Status 
information for unpublished applications is available through Private PAIR only. For 
more information about the PAIR system, see http://pair-direct.uspto.gov. Should you 
have questions on access to the Private PAIR system, contact the Electronic Business 
Center (EBC) at 866-217-9197 (toll-free). 

James H. Blackwell 
03/17/2006 




